Weather workflow using Canberra as a
demo
Abstract
Understanding current and short term predictions of local weather is an important and often highly beneficial information stream to have for any small business owner. But curating and documenting data collection can be optismed using api calls and simple data workflows. This manuscript runs through a workflow to document metadata from a developing API database. We use Canberra daily weather reports for a landscaping business. With data and tools being applied for financial benefit such as xx and xx there is a need to make more accessible applications and data sources for all small businesses.
This manuscript creates a computationally reproducible workflow that generates the following figures and interactive documents from our single github repository:
Australia’s National Bureau of Weather (BOM).
This is another option to the BOM data and provides a series of variables with similar attributes as the BOM API. In future these sections of code maybe helpful with generation of automatic conversion from a single format to another….
First we need to join the information by the same date and time.
Total precipitation can be worked with in several ways. From real rain radar to predictions of rain in the 3hour period (not sure if it is before or after at this point (jUne 2022)).
Multiple different organisations provide predictions of the Canberra weather. Here I have combined these predictions into a general estimate of weather from each forecast for different time incriments. To do this the coding needs to take each prediction and collate the relevant information into a structure that allows us to compare different versions of the weather prediction along a timeline of Australian times and dates….ACT times and dates sorry….
##
## Attaching package: 'maps'
## The following object is masked from 'package:purrr':
##
## map
#access new data
source(here::here("R/additional_code/bomrangRADAR.R"))
## file downloaded to:data/BOMradarIMG/radar.png
#visualise images stored
#xx
# get_radar_imagery(product_id = "IDR403")
Here we have taken advantage of the tools and other available tools for open science to create a dynamic dashboard for a local landscaping company. The outcome of the project is a simple interactive dashboard including mapping and predicting of the future weather from several different data sources. The ability to then extend this work into a commercially viable structure is still a long way away. To demonstrate this we have created a reproducible workflow that is curated in RMarkdown and published to github for free to share. With the development of these open source tools we hope to make it apparent that more application within the R and particular the RMarkdown environment to help facilitate both small business growth and scientific discovery.
Although the tools we have demonstrated here apply to simple, small scale (non-viral) applications. There are many examples of these sorts of applications publicly (cite). This can only indicate the number also applied in private. If this is the case the increase advantages of these tools may result in only sections of the social and global community having access to such sources. There is evidence of this in
include_graphics(here::here("graphics/weatherMap.png"))
## Warning in include_graphics(here::here("graphics/weatherMap.png")): It is highly
## recommended to use relative paths for images. You had absolute paths: "D:/
## Weather2022/graphics/weatherMap.png"
The study area for this data is the ACT region. We have divided this area up into unique areas that the landscaping business does regular jobs. This allows us to generate locally specific information with some understanding of the confidence of the evidence based on the proximity of the location to the weather station and forecasting information. Figure 1 represents the areas and divisions we have used to group our data as we update the information using our R code at regular intervals.
The approach to estimating and developing models for real-time data estimation and prediction has many applications and developments. Here we combine methods from the biological, computational and social science fields to make the most of the data aval;iable at the same time as accounting for bias in the collection, visualisation and communication of this information. We choose to use weather as a example as the field is developing fast with the additional data from satellites and other devices in communities allowing for much better response times in dangerous situations such as war and natural disasters.
We collect weather data from the following sources:
#package info current check
# BOM
# OWM_API_KEY
# Sys.setenv(OWM_API_KEY = OWM_API_KEY)
library(owmr)
current <- get_current("Canberra", units = "metric", lang = "en")
class(current)
## [1] "list" "owmr_weather"
forecast <- get_forecast("Canberra", units = "metric")
# class(forecast)
cbr <- search_city_list("Canberra")
current_multiple <- find_cities_by_geo_point(
cbr$lat,
cbr$lon,
cnt = 5,
units = "metric"
)
# class(current_multiple)
We then use functions and tidy data wrangling tools (dplyr etc) to generate a standard weather dataset for this flexdashboard and application.
#create visdiagram
knitr::include_graphics("graphics/weatherdataFIG-page-0.png")
This will then allow the product to be passed to the landscaping company for testing and further development.
The below section deals with data collection from image data such as satillites and gif’s from web pages. We collect image data from the following sources:
#latest dataset
tab3 <- read.csv("https://raw.githubusercontent.com/green-striped-gecko/covid_canberra/main/data/last.csv")
# Aggregate method
cols <- c( "red", "yellow","blue")
addBuses <- FALSE
tab3$Contact <- factor(tab3$Contact,levels=c( "Close" , "Casual", "Monitor") )
cc <- as.numeric( tab3$Contact)
ncols <- c("black","cyan")
nn <- as.numeric(factor(tab3$Status))
nn2 <- ifelse(nn==1,nn, 3)
tab3$Date <- substr(tab3$Date,1,10) #ditch day of the week
tab3$Date <- as.Date(tab3$Date, format = "%d/%m/%Y")
labs <- paste(tab3$Contact, tab3$Status,tab3$Exposure.Location, tab3$Street, tab3$Suburb, tab3$Date,tab3$Arrival.Time, tab3$Departure.Time, tab3$doubles, sep="<br/>")
###############################################
##plot the map
df_shared <- crosstalk::SharedData$new(tab3[,c(1,8,2:7)])
m <- leaflet(df_shared) %>% addTiles()
m
#map of stations and data sources
# library()
But by adding additional data sources we are able to increase the number of weather stations reporting to this dataset using:
This report generates a collection of images and data needed to distribute and inform userrs of the data and updates.
Our curated dataset allows for the aims of our project meet and have resulted in the following baseline set of variables. To create the dataset above there are several important structural choices that have been made based on the overall project goals and the other avaliable tools.
# table of metadata about variables
First of all you have to store your API key in an environment
variable called OWM_API_KEY. For the current session this
can be done with:
OWM_API_KEY
Sys.setenv(OWM_API_KEY = OWM_API_KEY)
Due to the ... parameter in R you can pass all
parameters defined in the openweathermap-api-reference
to the functions fetching data.
library(owmr)
current <- get_current("Canberra", units = "metric", lang = "en")
class(current)
## [1] "list" "owmr_weather"
forecast <- get_forecast("Canberra", units = "metric")
# class(forecast)
cbr <- search_city_list("Canberra")
current_multiple <- find_cities_by_geo_point(
cbr$lat,
cbr$lon,
cnt = 5,
units = "metric"
)
class(current_multiple)
## [1] "list" "owmr_find"
# bbox_greater_london <- c(-0.489, 51.28, 0.236, 51.686, 10)
# current_greater_london <- find_cities_by_bbox(bbox_greater_london)
# class(current_greater_london)
# current_greater_cbr %>% owmr_as_tibble() %>%
# .[, c(columns, "name")]
owmr_as_tibblecolumns <- c("dt_txt", "temp","pressure", "humidity" ,"temp_min", "temp_max")
current %>% owmr_as_tibble() %>%
.[, columns]
forecast %>% owmr_as_tibble() %>%
.[1:4, columns]
current_multiple %>% owmr_as_tibble() %>%
.[, c(columns, "name")]
cbr <- search_city_list("Canberra")
# Get data for 9 cities around cbr
owm_data <- find_cities_by_geo_point(
lat = cbr$lat,
lon = cbr$lon,
cnt = 9,
units = "metric"
) %>% owmr_as_tibble()
owm_data %<>% parse_columns(list(temp = round))
head(owm_data)
# Create a popup template
popup_tpl <- paste0(
"<b>{{name}}</b></br>",
"{{coord_lon}}, {{coord_lat}}</br>",
"{{temp}}°C, ",
"<i>{{weather_description}}</i>"
)
# Test it ...
popup_tpl %$$% owm_data %>% head(2)
## [1] "<b>Canberra</b></br>149.1281, -35.2835</br>6°C, <i>broken clouds</i>"
## [2] "<b>Russell</b></br>149.151, -35.2977</br>5°C, <i>light intensity shower rain</i>"
library(leaflet)
leaflet(width = "100%") %>%
addProviderTiles("CartoDB.DarkMatter") %>%
add_weather(owm_data, icon = owm_data$weather_icon, template = popup_tpl)
##snapshot weather
Furthermore, you can add weather map layers from OpenWeatherMap to
the map object via add_owm_tiles. For
available layers see owm_layers.
From static to dynamic documents and back again using rmarkdown and knitr…..
To combine the data for this project the following functions have been developed:
Coming soon…
Canberra is:
Landscaping and garden works are common locally based small businesses that are heavily impacted by local weather conditions. We use this case study to create a dynamic dashboard of curated locations and there association weather data for a morning (7am report). For a single live site see xx [coming soon]
to reproduce dashboard run the following code…
I will create a set of unique weather icons in the future. For now
owm_data$weather_icon
## [1] "04n" "09n" "09n" "09n" "09n" "04n" "04n" "04n" "09n"
References should follow the ACM standard. The example provided here uses the class file and bib style file. For example, we could write that published a review on EDM in this journal; other reviews were published later . The provided ref.bib file contains examples of virtually every possible citation type.
Supplementary technical material (e.g., mathematical proofs,descriptions of experimental procedures and additional code and notes).
Supplementary material should be collected in an appendix at the end of the paper (before the acknowledgements and the references sections).
Each figure and table must be mentioned in the text, and must be numbered consecutively in order of appearance (with captions in lower case). For the review process, the figures should be integrated into the text rather than being inserted at the end of the document.
As JEDM is an electronic journal, authors are encouraged to use colours when it enhances visibility and understanding of figures.
# source("R/openLeafMap.R")
# leaf1
Examples of a figure and a table are given in Figure~\(\ref{tab:fig1}\) and Table~\(\ref{tab:1}\).
For LaTeX, a style file named is provided on the journal’s web site. An MS Word file containing this example text is also provided, , which can be used as a template (see the Styles menu). Note that copy/pasting into the template may not ensure correct formatting but selecting a region of text and applying the appropriate style should always work.
Footnotes should be used sparingly and indicated by consecutive superscript numbers in the text. Material to be footnoted should appear at the bottom of the page on which it is referenced. Acknowledgments and grant numbers should be put into a separate ‘Acknowledgment’ section right before the list of references.
Support our work here:
anthony.davidson@canberra.edu.au
Comments per Day